Thirty Days of Metal — Day 20: Multisample Antialiasing

Warren Moore

·

Follow

7 min read

·

Apr 29, 2022

15

This series of posts is my attempt to present the Metal graphics programming framework in small, bite-sized chunks for Swift app developers who haven’t done GPU programming before.

If you want to work through this series in order, start here. To download the sample code for this article, go here.

When rendering, the resolution of our virtual canvas can be an important performance consideration: the more pixels we have to shade, the longer our frames take to process. So far, we have been using MTKView’s default behavior, which determines the size of drawables by multiplying the view’s bound’s size by the layer’s contents scale. On a Retina display, this tends to create drawable textures with many millions of pixels.

One option we have to reduce render target memory and increase performance is to render at a lower resolution by manually controlling the drawable size. For example, we might choose to render at a fixed vertical resolution of 1080 pixels and allow the system to “upscale” the result to the appropriate size by bilinear filtering. This manual approach is frequently used by console games, which require consistent framerates and have constrained memory budgets.

The downside to using smaller render targets, of course, is that we can introduce artifacts, such as “jaggies,” visible stairstep patterns that result from not having enough pixels to represent smooth geometric outlines.

One response to such aliasing is supersampling, which renders multiple samples per pixel and averages the result to produce a smoother image. We call the number of samples evaluated per pixel the sample count. If the total number of samples rendered by reducing the nominal resolution and taking multiple samples per pixel reduces the overall workload, supersampling can be a reasonable choice. However, there is often a better alternative.

Multisample Antialiasing

Multisample antialiasing (MSAA) is an antialiasing technique that exploits a particular fact of geometry to reduce the workload below that of supersampling at an equivalent sample count. Specifically, MSAA takes advantage of the fact that jaggies are most prone to occur on the silhouette edges of primitives. Within a primitive, rasterization tends to produce smooth results, but at primitive edges, a primitive either covers a pixel or it doesn’t.

MSAA works by taking a single sample for each pixel inside a primitive and multiple subpixel samples along primitive edges. The samples are stored in a multisample rendering target, which takes up memory proportional to the resolution multiplied by the sample count. MSAA render targets can be much larger than their single-sampled counterparts; the savings come from the intelligent sampling along primitive edges.

Below is an example of the effects of MSAA on a rendered image. Note that in the left image, silhouette edges are quite jagged, while in the the right image, which uses 4 MSAA samples per pixel, the edges are much more smoothly blended.

In order to turn a multisampled render texture into an image that can be displayed, we must resolve it into a single-sampled texture. This process happens at the end of a pass after rendering is complete, and is configured on the render pass descriptor.

Sample Counts and Positions

Increasing the sample count tends to increase image quality; that’s why we’re using MSAA in the first place. But each additional sample also occupies additional texture memory (at least on some GPUs). Additionally, most GPUs support a fixed maximum sample count. In Metal, the supported sample counts tend to be powers of 2: 1, 2, 4, and 8.

Each sample count has a corresponding default arrangement of the subpixel sample positions. The sample positions for 2x, 4x, and 8x MSAA are shown in the figure below.

MSAA with MTKView

The MTKView class is MSAA-aware: you can easily configure it to create MSAA render targets and resolve them automatically at the end of the render pass that renders into the view’s drawable.

You configure the view for MSAA by setting its sampleCount property. A common sample count is 4.

mtkView.sampleCount = 4

Setting this single property causes the MTKView to allocate MSAA-enabled color and depth textures on your behalf, and configure the render pass descriptors returned by the currentRenderPassDescriptor property to perform MSAA.

The only other change you are obligated to make to support MSAA is to inform the render pipeline creation process that you will be using MSAA. This is done by setting the rasterSampleCount property on the render pipeline descriptor:

let renderPipelineDescriptor = MTLRenderPipelineDescriptor()
renderPipelineDescriptor.rasterSampleCount = mtkView.sampleCount

Now, when rendering, the graphics pipeline will run the fragment shader multiple times per pixel, store the resulting colors in the multisampled textures created by the view, and resolve the MSAA textures into their single-sampled counterparts (including the drawable texture, which is then presented as usual).

Creating MSAA Render Targets Manually

Of course, this article series isn’t just about using MetalKit; it’s also about using Metal’s lower-level facilities. So let’s look at how to manually create render target textures and configure render pass descriptors to use them.

We add a few properties to our renderer class to support MSAA rendering:

let rasterSampleCount = 4
var msaaColorTexture: MTLTexture?
var msaaDepthTexture: MTLTexture?

All textures have a fixed size, so if and when the drawable size of the view changes, we need to throw away any existing MSAA targets and recreate them. Fortunately, we can respond to such changes in the mtkView(_:drawableSizeWillChange:) delegate method:

func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) {
    msaaColorTexture = nil
    msaaDepthTexture = nil
}

To recreate these textures on-demand, we add a method called makeMSAARenderTargetsIfNeeded() that is called at the top of the draw method.

First of all, we don’t need to create MSAA targets if MSAA isn’t being used. We check this by testing whether the renderer’s sample count is greater than 1:

func makeMSAARenderTargetsIfNeeded() {
    if rasterSampleCount == 1 { return }

We only need to recreate render targets if (1) they haven’t been created previously, (2) their size differs from our drawable size, or (3) their sample count differs from our desired sample count.

We first convert the drawable width and height to integers:

let drawableWidth = Int(view.drawableSize.width)
let drawableHeight = Int(view.drawableSize.height)

Now we test our existing MSAA color target, if any, for the conditions listed above.

if msaaColorTexture == nil ||
   msaaColorTexture?.width != drawableWidth ||
   msaaColorTexture?.height != drawableHeight ||
   msaaColorTexture?.sampleCount != rasterSampleCount

In the event we do need to create MSAA targets, we use the now-familiar procedure of filling out a texture descriptor.

let textureDescriptor = MTLTextureDescriptor()
textureDescriptor.textureType = .type2DMultisample
textureDescriptor.sampleCount = rasterSampleCount
textureDescriptor.pixelFormat = view.colorPixelFormat
textureDescriptor.width = drawableWidth
textureDescriptor.height = drawableHeight
textureDescriptor.storageMode = .private
textureDescriptor.usage = .renderTarget

The chief differences between this descriptor and others we have seen are the texture type, which is set to MTLTextureType.type2DMultisample, and the sampleCount property, which is set to match the preferred MSAA sample count. Since the texture will act as a render target and does not need to be accessible to the CPU, we set its usage and storage mode accordingly.

Then we can create and store the MSAA color texture:

msaaColorTexture = device.makeTexture(descriptor: textureDescriptor)

The procedure for creating the depth texture is almost identical, so I won’t repeat it here. Instead of using the view’s colorPixelFormat property to select the depth pixel format, we use its depthStencilPixelFormat.

Creating MSAA Render Pass Descriptors

Since we are now creating MSAA targets manually, we do not need to inform MTKView that we are using MSAA; we can leave the view’s sampleCount set to 1. However, this means we are responsible for creating our own render pass descriptors, since the one returned by currentRenderPassDescriptor will be invalid for MSAA rendering.

We will write a method, renderPassDescriptor(colorTexture:,depthTexture:) that handles both the MSAA and non-MSAA cases.

First, we instantiate the render pass descriptor:

func renderPassDescriptor(colorTexture: MTLTexture,
                          depthTexture: MTLTexture?)
                          -> MTLRenderPassDescriptor
{
    let renderPassDescriptor = MTLRenderPassDescriptor()

For each render target that has an MSAA target, we need to configure the corresponding render pass attachment with the MSAA texture and a “resolve texture” to store the final results.

We begin by checking whether MSAA is enabled and setting up the MSAA targets if so:

let msaaEnabled = (rasterSampleCount > 1)if msaaEnabled {
    renderPassDescriptor.colorAttachments[0].texture = msaaColorTexture
    renderPassDescriptor.colorAttachments[0].resolveTexture = colorTexture
    renderPassDescriptor.colorAttachments[0].loadAction = .clear
    renderPassDescriptor.colorAttachments[0].clearColor = view.clearColor
    renderPassDescriptor.colorAttachments[0].storeAction = .multisampleResolve    renderPassDescriptor.depthAttachment.texture = msaaDepthTexture
    renderPassDescriptor.depthAttachment.resolveTexture = depthTexture
    renderPassDescriptor.depthAttachment.loadAction = .clear
    renderPassDescriptor.depthAttachment.clearDepth = view.clearDepth
    renderPassDescriptor.depthAttachment.storeAction = .multisampleResolve
}

The thing to pay attention to here is the store action. For both the color and depth attachment, the store action is MTLStoreAction.multisampleResolve, which tells Metal to average the multiple samples for each pixel down to a single value to write into the resolve texture.

The non-MSAA case is easier to handle and should look familiar from our explorations with shadow mapping (the only exception being that we discard the contents of the depth map at the end of the pass):

else {
    renderPassDescriptor.colorAttachments[0].texture = colorTexture
    renderPassDescriptor.colorAttachments[0].loadAction = .clear
    renderPassDescriptor.colorAttachments[0].clearColor = view.clearColor
    renderPassDescriptor.colorAttachments[0].storeAction = .store    renderPassDescriptor.depthAttachment.texture = depthTexture
    renderPassDescriptor.depthAttachment.loadAction = .clear
    renderPassDescriptor.depthAttachment.clearDepth = view.clearDepth
    renderPassDescriptor.depthAttachment.storeAction = .dontCare
}

Finally, we return the configured render pass descriptor for use in our draw method:

return renderPassDescriptor
}

We can now use this method instead of asking the view for a render pass descriptor. We still use the drawable’s texture and the view’s internal depth texture; no reason to reinvent that particular wheel.

guard let drawable = view.currentDrawable else { return }
let renderPassDescriptor = renderPassDescriptor(
    colorTexture: drawable.texture,
    depthTexture: view.depthStencilTexture)

The difference afforded by MSAA may seem subtle, but at lower base resolutions, it can make all the difference between swimming jaggies and smooth sailing. Here’s a multisample antialiased version of the screenshot from last time:

Next time, we will look at point lights, to add more expressiveness to our lighting environments.